Skip to content

helm: support gRPC probes and label the service port as gRPC - #587

Draft
wseaton wants to merge 1 commit into
ai-dynamo:mainfrom
wseaton:chart-grpc-fixes
Draft

helm: support gRPC probes and label the service port as gRPC#587
wseaton wants to merge 1 commit into
ai-dynamo:mainfrom
wseaton:chart-grpc-fixes

Conversation

@wseaton

@wseaton wseaton commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

The chart hardcoded tcpSocket probe handlers and named the service/container port http. Under an Istio sidecar a tcpSocket probe handshakes with the sidecar and reports Ready even when the server is down, and the http port name makes Istio protocol detection treat the broker's gRPC as HTTP/1.1, breaking it in-mesh.

Changes:

  • Probe values now render verbatim (minus enabled), so grpc, httpGet, exec, and tcpSocket handlers all work. Defaults switch to the Kubernetes grpc probe against the server's gRPC health service, keeping the existing timing values.
  • Service and container port renamed to grpc, with appProtocol: grpc on the Service.
  • MODEL_EXPRESS_LOGGING_LEVEL renamed to MODEL_EXPRESS_LOG_LEVEL, the variable the server actually reads (modelexpress_common/src/envs.rs).

Backward compatible: existing values files with tcpSocket probes still render. Because Helm deep-merges overrides with the new grpc defaults, the template drops the default grpc handler when an override supplies its own handler, so old-style files do not produce a two-handler probe.

Validated with helm lint helm/ and helm template against defaults, test-values.yaml, values-development.yaml, values-production.yaml, values-local-storage.yaml, plus an old-style tcpSocket override.

The probe templates only rendered a tcpSocket handler, which an Istio
sidecar answers even when the server is down, and the service/container
port was named "http", so Istio protocol detection treated the broker's
gRPC traffic as HTTP/1.1. Render probe values verbatim (minus "enabled")
so grpc/httpGet/exec handlers work, default the probes to the gRPC
health service the server already implements, and rename the port to
"grpc" with appProtocol set. When an override supplies its own handler
the default grpc handler is dropped, since a probe allows only one.

Also rename MODEL_EXPRESS_LOGGING_LEVEL to MODEL_EXPRESS_LOG_LEVEL, the
env var the server actually reads.

Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Will Eaton <weaton@redhat.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant